Wearable electronic device having a light field camera

ABSTRACT

A method of authenticating a user of a wearable electronic device includes emitting light into a dorsal side of a forearm near a wrist of the user; receiving, using a light field camera, remissions of the light from the dorsal side of the forearm near the wrist of the user; generating a light field image from the remissions of the light; performing a synthetic focusing operation on the light field image to construct at least one image of at least one layer of the forearm near the wrist; extracting a set of features from the at least one image; determining whether the set of features matches a reference set of features; and authenticating the user based on the matching. In some embodiments, the method may further include compensating for a tilt of the light field camera prior to or while performing the synthetic focusing operation.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/132,241, filed Sep. 14, 2018, which is a nonprovisional of and claimsthe benefit under 35 U.S.C. § 119(e) of U.S. Provisional PatentApplication No. 62/564,916, filed on Sep. 28, 2017, the contents ofwhich are incorporated by reference as if fully disclosed herein.

FIELD

The described embodiments relate generally to a watch or otherelectronic device (e.g., another type of wearable electronic device).More particularly, the described embodiments relate to a wearableelectronic device having a light field camera, and to techniques forperforming bioauthentication of a user of the device from a dorsal sideof a forearm near a wrist of the user.

BACKGROUND

An electronic device may include a fingerprint sensor, a facialrecognition sensor, a retina scanner, or other form of bioauthenticationsensor. In some devices, such as a phone or tablet computer, abioauthentication sensor may be provided adjacent (or as part of) adisplay of the device. However, in a wearable electronic device such asa watch, there may be little or no room for providing abioauthentication sensor adjacent (or as part of) a display of thedevice. User authentication may therefore be provided by means of apassword or similar input.

SUMMARY

Embodiments of the systems, devices, methods, and apparatus described inthe present disclosure are directed to a watch or other electronicdevice (e.g., another type of wearable electronic device) having a lightfield camera. The light field camera may be used to image a forearm neara wrist of a user. The imaging may be performed from a dorsal side ofthe forearm. A synthetic focusing operation may be performed on a lightfield image obtained from the light field camera, to construct at leastone image of at least one layer of the forearm near the wrist. A set offeatures of the forearm near the wrist may be extracted from the atleast one image and compared to a reference set of features (e.g., ahair follicle pattern, a vascular pattern, a vein pattern, an arterypattern, a blood perfusion pattern in skin, a blood perfusion pattern intendons, a blood perfusion pattern in fascia, a tendon pattern, aconnective tissue pattern, a skin pigmentation pattern, a pore pattern,and/or a bone shape pattern obtained during a bioauthenticationenrollment process performed for the user). An operation (e.g., abioauthentication operation, a bioauthentication enrollment operation, asecure transaction operation, a health monitoring operation, or a healthassessment operation) may be performed in response to whether the set offeatures matches the reference set of features. In some embodiments, atilt of the light field camera with respect to the dorsal side of theforearm may be determined and compensated for, prior to or whileperforming the synthetic focus operation.

In a first aspect, the present disclosure describes a watch body. Thewatch body includes a housing, a cover mounted to the housing, a lightemitter, a light field camera, and a processor. The cover has a firstsurface exterior to the watch body, and a second surface interior to thewatch body. The light emitter is positioned to emit light through thecover into a dorsal side of a forearm near a wrist of a user when thefirst surface of the cover is positioned adjacent the dorsal side of theforearm near the wrist of the user. The light field camera is positionedadjacent the second surface to receive remissions of the light throughthe cover from the dorsal side of the forearm near the wrist. Theprocessor is configured to operate the light emitter and the light fieldcamera, obtain a light field image from the light field camera, andperform a synthetic focusing operation on the light field image toconstruct at least one image of at least one layer of the forearm nearthe wrist.

In another aspect, the present disclosure describes another watch body.The watch body has a housing, a cover mounted to the housing, a lightemitter, a light field camera, and a tilt sensor. The cover has a firstsurface exterior to the watch body, and a second surface interior to thewatch body. The light emitter is positioned to emit light through thecover into a dorsal side of a forearm near a wrist of a user when thefirst surface of the cover is positioned adjacent the dorsal side of theforearm near the wrist. The light field camera is positioned adjacentthe second surface to receive remissions of the light through the coverfrom the dorsal side of the forearm near the wrist. The tilt sensor isconfigured to detect a tilt of the light field camera with respect tothe dorsal side of the forearm near the wrist.

In still another aspect of the disclosure, a method of authenticating auser of a wearable electronic device is described. The method includesemitting light into a dorsal side of a forearm near a wrist of the user;receiving, using a light field camera, remissions of the light from thedorsal side of the forearm near the wrist of the user; generating alight field image from the remissions of the light; performing asynthetic focusing operation on the light field image to construct atleast one image of at least one layer of the forearm near the wrist;extracting a set of features from the at least one image; determiningwhether the set of features matches a reference set of features; andauthenticating the user in response to the set of features matching thereference set of features.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thedrawings and by study of the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will be readily understood by the following detaileddescription in conjunction with the accompanying drawings, wherein likereference numerals designate like structural elements, and in which:

FIGS. 1 and 2 show an example of a watch including a biosensor system;

FIG. 3 shows the watch of FIGS. 1 and 2 attached to a forearm of a usernear the user's wrist;

FIGS. 4 and 5 show various features that may be identified in a set ofone or more layer images constructed from a light field image obtainedfrom a dorsal side of a forearm of a user near the user's wrist;

FIGS. 6-8 show a set of components that may be included in a biosensorsystem such as the biosensor system shown in FIGS. 2 and 3;

FIG. 9 shows the conical fields of view of some of the micro-camerasshown in FIGS. 6-8, as well as the feature of the forearm shown in FIG.8;

FIG. 10 shows an exploded view of another set of components that may beincluded in a biosensor system such as the biosensor system shown inFIGS. 2 and 3;

FIG. 11 shows an exploded view of another set of components that may beincluded in a biosensor system such as the biosensor system shown inFIGS. 2 and 3;

FIG. 12 shows a watch attached to, and tilted on, a forearm of a usernear the user's wrist;

FIG. 13 shows an alternative embodiment of the watch described withreference to FIGS. 1-3, in which a tilt sensor is included in the watch;

FIG. 14 shows the conical fields of view of some of the micro-camerasshown in FIGS. 6-8, as well as the feature of the forearm shown in FIG.8;

FIG. 15 shows a sample electrical block diagram of an electronic devicesuch as a watch or other wearable electronic device; and

FIG. 16 shows an example method of authenticating a user of a watch orother wearable electronic device.

The use of cross-hatching or shading in the accompanying figures isgenerally provided to clarify the boundaries between adjacent elementsand also to facilitate legibility of the figures. Accordingly, neitherthe presence nor the absence of cross-hatching or shading conveys orindicates any preference or requirement for particular materials,material properties, element proportions, element dimensions,commonalities of similarly illustrated elements, or any othercharacteristic, attribute, or property for any element illustrated inthe accompanying figures.

Additionally, it should be understood that the proportions anddimensions (either relative or absolute) of the various features andelements (and collections and groupings thereof) and the boundaries,separations, and positional relationships presented therebetween, areprovided in the accompanying figures merely to facilitate anunderstanding of the various embodiments described herein and,accordingly, may not necessarily be presented or illustrated to scale,and are not intended to indicate any preference or requirement for anillustrated embodiment to the exclusion of embodiments described withreference thereto.

DETAILED DESCRIPTION

Reference will now be made in detail to representative embodimentsillustrated in the accompanying drawings. It should be understood thatthe following description is not intended to limit the embodiments toone preferred embodiment. To the contrary, it is intended to coveralternatives, modifications, and equivalents as can be included withinthe spirit and scope of the described embodiments as defined by theappended claims.

As previously mentioned, a wearable electronic device such as a watchmay have little or no room for providing a biometric authentication(i.e., bioauthentication) sensor adjacent (or as part of) a display ofthe device. In a watch, a sensor can obtain measurements of biologicalparameters from the backside of the watch. However, bioauthenticationvia a backside sensor is complicated by the pattern, type, and densityof a user's arm hair, which tends to be longer and more dense on thedorsal side of the forearm where a backside of a watch normally sits.Hair also tends to have a highly variable position, which can make itdifficult to use for purposes of bioauthentication.

One type of sensor that could be used for bioauthentication via thebackside of a watch is a vein pattern sensor (e.g., a camera). However,veins and arteries and other significant structures of the forearm nearthe wrist tend to be deep beneath the surface (skin) of the forearm. Insome cases, light may emitted into a user's skin via the backside of awatch, and remitted (e.g., reflected or refracted) from features withina user's forearm near their wrist. However, light remitted from featuresdeep within the forearm can be blurred by scattering as it propagatestoward the surface of the skin Remitted light may also be scattered byarm hair. Depending on the user, a typical image sensor (or camera)positioned on the backside of a watch may receive little remitted light,and/or a processor may be unable to identify features of a user'sforearm based on the light remitted from features of the forearm.

Another issue with performing bioauthentication using a sensor on thebackside of a watch is that the footprint of the sensing area may belimited by the dimensions of the watch body, which can restrict theamount of data that an image sensor is able to acquire.

Wearable electronic devices and techniques described herein utilize abiosensor system including a light field camera. In such a system,light, such as infrared light, is emitted deep within a user's forearm.Remission of the light is captured by a light field camera. A lightfield camera is a camera that not only captures the intensity ofreceived light, but the intensities of light received in particularlight rays (or small sets of light rays) received from differentdirections. In some embodiments, a light field camera may include anarray of micro-cameras, with each micro-camera having an image sensingregion, which image sensing region has a limited field of view definedby a pinhole or microlens. The pinhole or microlens limits the imagesensing region's field of view to a set of light rays that passesthrough the same pinhole or microlens, but from different directions.Thus, each pixel value of the image sensing region can be associatedwith directional information based on the relationship of the pinhole ormicrolens to particular pixels of the image sensing region. A lightfield image may include a set of images acquired by the differentmicro-cameras in the light field camera (or a set of images acquired bya subset of the micro-cameras). The fields of view (or targets) of eachmicro-camera overlap, and the resolution of a light field camera can beincreased by configuring the fields of view to have significant overlap(e.g., in some embodiments, there may be a 50%, 75%, 90%, or greateroverlap between the fields of view of adjacent micro-cameras).

A synthetic focusing operation (e.g., a tomographic-style focusingoperation) may be performed on a light field image. The syntheticfocusing operation may constructively and deconstructively combine thepixel values of different images (e.g., elemental images) within a lightfield image, to construct images of layers at different distances fromthe sensing plane (i.e., images of different layers of a user'sforearm). A variety of exemplary synthetic focusing operations arewell-known in the art and may be based on shifting, scaling, adding,subtracting, and averaging pixel values. A processor may thereforeextract, from the images of one or more layers constructed during asynthetic focusing operation, a set of features of a user's forearm. Theset of features may include, for example, a hair follicle openingpattern, a hair follicle pattern, a vascular pattern, a vein pattern, anartery pattern, a blood perfusion pattern in skin, a blood perfusionpattern in tendons, a blood perfusion pattern in fascia, a tendonpattern, a connective tissue pattern, a skin pigmentation pattern, asmall scale folding pattern of skin, a pore opening pattern, a porepattern, and/or a bone shape pattern.

Synthetic focusing performed on a light field image may not only be usedto extract features of a user's forearm from the light field image, maybe used to classify the features based on size, depth, color, movement,variance between layers, existence in one or more layers, relationshipswith other features (e.g., relationships based on size, depth, color, ormovement with respect to other features), and so on. Although syntheticfocusing may not fully remove the blurring caused by turbid biologicalmaterials and light scattering as remitted light propagates toward auser's skin from deeply located structures, synthetic focusing mayde-emphasize features/patterns located in front of otherfeatures/patterns, which close features/patterns might otherwise obscuredeeper features/patterns.

In some embodiments, a processor that performs a synthetic focusingoperation may utilize a hardware accelerator to perform part or all ofthe synthetic focusing operation.

These and other embodiments are discussed with reference to FIGS. 1-16.However, those skilled in the art will readily appreciate that thedetailed description given herein with respect to these figures is forexplanatory purposes only and should not be construed as limiting.

The present disclosure recognizes that personal information data,including the biometric data acquired using the presently describedtechnology, can be used to the benefit of users. For example, the use ofbiometric authentication data can be used for convenient access todevice features without the use of passwords. In other examples, userbiometric data is collected for providing users with feedback abouttheir health or fitness levels. Further, other uses for personalinformation data, including biometric data, that benefit the user arealso contemplated by the present disclosure.

The present disclosure further contemplates that the entitiesresponsible for the collection, analysis, disclosure, transfer, storage,or other use of such personal information data will comply withwell-established privacy policies and/or privacy practices. Inparticular, such entities should implement and consistently use privacypolicies and practices that are generally recognized as meeting orexceeding industry or governmental requirements for maintaining personalinformation data private and secure, including the use of dataencryption and security methods that meets or exceeds industry orgovernment standards. For example, personal information from usersshould be collected for legitimate and reasonable uses of the entity andnot shared or sold outside of those legitimate uses. Further, suchcollection should occur only after receiving the informed consent of theusers. Additionally, such entities would take any needed steps forsafeguarding and securing access to such personal information data andensuring that others with access to the personal information data adhereto their privacy policies and procedures. Further, such entities cansubject themselves to evaluation by third parties to certify theiradherence to widely accepted privacy policies and practices.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data, including biometric data. That is, thepresent disclosure contemplates that hardware and/or software elementscan be provided to prevent or block access to such personal informationdata. For example, in the case of biometric authentication methods, thepresent technology can be configured to allow users to optionally bypassbiometric authentication steps by providing secure information such aspasswords, personal identification numbers (PINS), touch gestures, orother authentication methods, alone or in combination, known to those ofskill in the art. In another example, users can select to remove,disable, or restrict access to certain health-related applicationscollecting users' personal health or fitness data.

FIGS. 1 and 2 show an example of a watch 100 including a biosensorsystem 116. The watch 100 may include a watch body 102 and a watch band104. The watch 100 and watch body 102 are examples of wearableelectronic devices. Other wearable electronic devices that may include abiosensor system, such as one of the biosensor systems described in thepresent disclosure, include health monitoring or fitness devices,portable computing devices, mobile phones (including smart phones), ordigital media players. In some embodiments, the functions of some or allof these other wearable electronic devices may be incorporated into awatch, such as the watch 100.

The watch body 102 may include a housing 106. The housing 106 mayinclude one or more housing members. A singular housing member is shownin FIG. 1. The housing 106 may be metallic, plastic, ceramic, orcrystalline, or may include a combination of such materials, or mayinclude other materials.

A cover 108 may be mounted to the housing 106 on a front side of thewatch body 102 (i.e., facing away from a user's skin), as shown inFIG. 1. The cover 108 may protect a display within the housing 106 (andin some cases, the display may be mounted partly or wholly to the cover108). The display may be viewable by a user through the cover 108. Insome cases, the cover 108 may be part of a display stack, which displaystack may include a touch sensing or force sensing capability. Thedisplay may be configured to depict a graphical output of the watch 100,and a user may interact with the graphical output (e.g., using a finger,stylus, crown 110, or button 112). As one example, the user may select(or otherwise interact with) a graphic, icon, or the like presented onthe display. For example, the user may interact with a graphic on thedisplay by touching or pressing on the display at the location of thegraphic. The cover 108 may be considered separate from the housing 106,or alternatively, the cover 108 may be considered a component (e.g., ahousing member) of the housing 106. In some examples, the cover 108 maybe a crystal, such as a sapphire crystal. The cover 108 mayalternatively be formed of glass, plastic, or another material (ormaterials) that is transmissive to at least one wavelength of light(e.g., visible light).

Another cover 114 may be mounted to the housing 106 on a back side ofthe watch body 102 (i.e., facing a user's skin), as shown in FIG. 2. Thecover 114 may protect part or all of a biosensor system 116 mountedwithin the housing 106 (and in some cases, the biosensor system 116 maybe mounted partly or wholly to the cover 114). The biosensor system 116may include a light field camera positioned adjacent the cover 114 toreceive light through the cover 114. The biosensor system 116 may beconfigured to image a forearm of a user near a wrist of the user, andperform a bioauthentication of the user based on the imaging. The cover114 may be considered separate from the housing 106, or alternatively,the cover 114 may be considered a component (e.g., a housing member) ofthe housing 106. In some examples, the cover 114 may be a crystal, suchas a sapphire crystal. The cover 114 may alternatively be formed ofglass, plastic, or another material that is transmissive to at least onewavelength of light (e.g., visible light, such as infrared light, orinvisible light (i.e., other forms of electromagnetic radiation)).

The watch body 102 may include at least one input device or selectiondevice, such as a crown assembly, scroll wheel, knob, dial, button, orthe like, which input device may be operated by a user of the watch 100.For example, the housing 106 may include an aperture through which ashaft extends. A crown 110 may be attached to the shaft, and may beaccessible to a user exterior to the housing 106. The crown 110 may bemanipulated by a user to rotate or translate the shaft. The shaft may bemechanically, electrically, magnetically, and/or optically coupled tocomponents within the housing 106 as one example. A user's manipulationof the crown 110 and shaft may be used, in turn, to manipulate or selectvarious elements displayed on the display, to adjust a volume of aspeaker, to turn the watch 100 on or off, and so on. The housing 106 mayalso include an aperture through which a button 112 protrudes.

The housing 106 may include structures for attaching the watch band 104to the watch body 102. In some cases, the structures may includeelongate recesses or apertures through which ends of the watch band 104may be inserted and attached to the watch body 102. In other cases (notshown), the structures may include indents (e.g., dimples ordepressions) in the housing 106, which indents may receive ends ofspring pins that are attached to or threaded through ends of a watchband to attach the watch band to the watch body.

The watch band 104 may be used to secure the watch 100 to a user,another device, a retaining mechanism, and so on.

In some examples, the watch 100 may lack the cover 108, the display, thecrown 110, or the button 112. For example, the watch 100 may include anaudio input or output interface, a touch input interface, a haptic(force) input or output interface, or other input or output interfacethat does not require the display, crown 110, or button 112. The watch100 may also include the afore-mentioned input or output interfaces inaddition to the display, crown 110, or button 112. When the watch 100lacks the display, the front side of the watch 100 may be covered by thecover 108, or by a metallic or other type of housing member.

In some embodiments, each of the covers 108, 114 may include anytransparent, semi-transparent, or translucent surface made out of glass,a crystalline material (such as sapphire or zirconia), plastic, or thelike, has and may have a crystal or non-crystalline atomic structure.

Turning now to FIG. 3, the watch 100 is shown attached to a forearm 300of a user near the user's wrist. Attachment of the watch 100 to theforearm 300 near the wrist may position a first or exterior surface ofthe cover 114 (i.e., a surface exterior to the watch body 102) adjacenta dorsal side of the forearm 300 near the user's wrist.

The biosensor system 116 may include a light emitter (or set of lightemitters) positioned to emit light into the dorsal side of the forearm300 near the wrist. The biosensor system 116 may also include a lightfield camera positioned within the watch body 102, adjacent a second orinterior surface of the cover 114 (i.e., a surface interior to the watchbody 102). The light field camera may receive, from the dorsal side ofthe forearm 300 near the wrist, remissions of the light emitted by thelight emitter(s). The remissions of light may include reflections andrefractions of the light by features on or within the forearm 300 of theuser near the user's wrist (e.g., features of skin, hair, hairfollicles, pores, vascular structures, connective tissues, bones, and soon).

A processor (e.g., a processor within the watch body 102) may beconfigured to operate the light emitter(s) and light field camera toobtain a light field image from the light field camera. In someembodiments, the light field camera or another sensor of the watch 100(e.g., a proximity sensor) may detect when the watch 100 is positionedon and/or attached to the user's forearm 300 and trigger the processorto operate the light emitter(s) and light field camera, obtain a lightfield image from the light field camera, and perform an operation suchas a bioauthentication operation, a bioauthentication enrollmentoperation, a secure transaction operation, a health monitoringoperation, or a health assessment operation using the light field image.In some embodiments, a utility or application running on the watch 100(or a utility or application running remote from the watch 100) maytrigger the processor to operate the light emitter(s) and light fieldcamera, obtain a light field image from the light field camera, andperform an operation such as a bioauthentication operation, abioauthentication enrollment operation, a secure transaction operation,a health monitoring operation, or a health assessment operation usingthe light field image.

In some examples, the light field camera of the biosensor system 116 mayinclude a set of micro-cameras having overlapping fields of view (i.e.,an array of micro-cameras). Each camera may be operable to acquire animage (e.g., an elemental image), which image may include a plurality ofpixel values. Each pixel value may indicate the intensity of a light rayremitted from a particular direction (or in practice, the intensity of asmall number of light rays remitted from a small set of directions).Associations between pixel values and light ray directions may beestablished by an array of light-transmissive elements such as pinholesor microlenses, which light-transmissive elements restrict the aperturesof the micro-cameras and/or direct rays of light remitted fromparticular directions to particular pixels of the micro-cameras.

The light field image obtained by the processor may include a set ofimages (e.g., a set of elemental images) acquired by some or all of themicro-cameras of a light field camera. The processor may be configuredto perform a synthetic focusing operation on the light field image toconstruct images of one or more layers of the user's forearm 300 nearthe user's wrist (e.g., an image of at least one layer in thethree-dimensional space 302 defined by the overlapping fields of view ofthe micro-cameras of the light field camera in the biosensor system 116;see FIGS. 2 and 3). Performance of the synthetic focusing operation mayentail combining pixel values in the set of images in different ways toconstruct images of different layers of the user's forearm 300.

The processor may be configured to extract a set of features from theimage(s) constructed during the synthetic focusing operation. The set offeatures may include, for example, features of skin, hair, hairfollicles, pores, vascular structures, connective tissues, bones, and soon, as described in greater detail with reference to FIGS. 4 and 5. Insome embodiments, the processor may compare the set of features to areference set of features, to determine whether the set of featuresmatches the reference set of features. When the reference set offeatures are features of an authorized user (e.g., an authorized user ofthe watch 100, of a service provided by the watch (e.g., a payment,banking, or other secure transaction service), etc.), and when the setof features matches the reference set of features, the processor may,for example, authenticate the user of the watch 100 and enable the userto access the watch 100, or access a function of the watch 100, orcomplete a secure transaction. When the reference set of features arefeatures associated with a health status or condition, and when the setof features matches the reference set of features, the processor may,for example, indicate that the user of the watch 100 may have theassociated health status or condition, or provide an alert to medicalpersonnel. When the biosensor system 116 is operated as a part of abioauthentication enrollment operation, the processor may store the setof features identified from the image(s) constructed during thesynthetic focusing operation, so that the set of features may beaccessed at a later time as a reference set of features (e.g., as a setof features associated with an authorized user, or as a set of featuresassociated with a baseline health or condition of the user).

FIG. 4 shows various features that may be identified in a set of one ormore layer images constructed from a light field image obtained from adorsal side of a forearm near a wrist. The features may be associatedwith structures at or just beneath the surface of a user's skin (e.g.,on or into the epidermis 400), or structures that start at the surfaceof the user's skin and extend deeper beneath the user's skin (e.g., intothe dermis 402).

Examples of features associated with structures at or just beneath thesurface of a user's skin include a vascular pattern 404 (e.g., a bloodperfusion pattern in skin), a pigmentation pattern 406 (e.g., a skin (ormelanin) pigmentation pattern), a small scale folding pattern of skin, ahair follicle opening pattern 408, and a pore opening pattern 410. Ablood perfusion pattern in skin (e.g., a capillary blood perfusionpattern) may vary significantly with temperature, exercise, and otherbiological or environmental variables, and may have limited use forbiometric identification. A skin pigmentation pattern may be more stablethan a blood perfusion pattern in skin, but may vary with exposure toultraviolet light (e.g., sunlight).

Examples of features associated with structures that start at thesurface of a user's skin and extend beneath the user's skin include ahair follicle pattern 412, and a pore pattern 414 (e.g., sweat ducts).In some cases, hair follicles may be identified by first identifyingpoints at which hair intersects (or enters) the epidermis (e.g., a hairfollicle opening pattern 408 in one or more shallow layers of aforearm), and then correlating these points with structures (e.g.,cross-sections of hair follicles) identified in deeper layers of aforearm. In some cases, sweat ducts may be identified based on thedifferent light absorption rates of sweat ducts and surrounding turbidtissues, which may contain different amounts of water and absorb light(e.g., infrared light) at different rates. Similarly to theidentification of hair follicles, pores may identified in surface orshallow layers of a forearm (e.g., in a pore opening pattern 410) andcorrelated with structures (e.g., cross-sections of sweat glands)identified in deeper layers of a forearm.

FIG. 5 shows additional features that may also or alternatively beidentified in a set of one or more layer images constructed from a lightfield image obtained from a dorsal side of a forearm 500 near a wrist502. The features may be associated with structures deep beneath thesurface of a user's skin (e.g., in subcutaneous tissue or bone).

Examples of features associated with structures deep beneath the surfaceof a user's skin include a vascular pattern 504 (e.g., a vein pattern,an artery pattern, a blood perfusion pattern in tendons, or a bloodperfusion pattern in fascia), a connective tissue pattern 506 (e.g., atendon pattern), and a bone shape pattern. A vascular pattern insubcutaneous tissue, and particularly a vein pattern or an arterypattern, may vary less than a vascular pattern in epidermal or dermaltissue. A vein pattern or an artery pattern, for example, may alsoinclude vein or artery cross-sectional size information that provides anextra level of detail and more biometric entropy than can be obtainedfor a capillary blood perfusion pattern.

FIGS. 6-8 show a set of components 600 that may be included in abiosensor system such as the biosensor system 116 shown in FIGS. 2 and3. FIGS. 6 and 8 show the components 600 in assembled form, and FIG. 7shows the components 600 in exploded form. The biosensor system includesa set of light emitters 602 and a light field camera 604, both of whichmay be positioned adjacent an interior surface of a cover 606. The cover606 may be the backside cover 114 that is mounted to the housing 106 ofthe watch body 102 shown in FIGS. 1-3.

The light emitters 602 may be positioned to emit light into a dorsalside of a forearm near a wrist of a user when an exterior surface of thecover 606 is positioned adjacent (e.g., on) the dorsal side of theforearm near the wrist. The light field camera 604 may be positioned toreceive emissions of the light (i.e., the light emitted by the lightemitters 602) from the dorsal side of the forearm near the wrist.

As shown in FIG. 7, the light field camera 604 may include a first arrayof non-overlapping image sensing regions 608, and a second array oflight-transmissive elements (e.g., an array of pinholes 610 defined by apinhole mask 612). The pinhole mask, including the light-transmissiveelements (e.g., pinholes), may be positioned between the array ofnon-overlapping image sensing regions 608 and the interior surface ofthe cover 606. In some embodiments, the pinhole mask 612 may include asubstrate made of opaque plastic or another opaque material. In someembodiments, the pinhole mask 612 may include patterned ink deposited onthe interior surface of the cover 606.

The array of non-overlapping image sensing regions 608 may be alignedwith the array of light-transmissive elements to form a set ofmicro-cameras. In some embodiments, and as shown, the array ofnon-overlapping image sensing regions 608 may include regions of asingle image sensor. Alternatively (and not shown), the array ofnon-overlapping image sensing regions 608 may include discrete imagesensors (e.g., one image sensor per image sensing region, or one imagesensor per subset of image sensing regions). The light field camera 604may optionally include a spacer 614 that separates (e.g., is between)the array of non-overlapping image sensing regions 608 and the array oflight-transmissive elements. In some embodiments, the spacer 614 mayinclude a layer of glass.

The light emitters 602 may be positioned around the light field camera604, and may emit light from around the light field camera 604. In someembodiments, the light emitters 602 may include light emitting diodes(LEDs). In some embodiments, the light emitters 602 may include infraredemitters.

The micro-cameras may have fields of view 616 that overlap on a side ofthe cover 606 opposite the light field camera 604, as shown in FIG. 8.In some cases, the fields of view 616 may begin to overlap within thecover 606. By way of example, the fields of view 616 may be conical.When the external surface of the cover 606 is positioned adjacent aforearm of a user near the user's wrist, the conical fields of view 616may extend into the forearm and define the extent of thethree-dimensional space 302 described with reference to FIG. 3.

As shown in FIG. 8, a feature 618 of a forearm may fall within thefields of view 616 of multiple micro-cameras. FIG. 9 shows the conicalfields of view 616 of some of the micro-cameras shown in FIGS. 6-8, aswell as the feature 618 of the forearm shown in FIG. 8. The lightemitters 602 and light field camera 604 described with reference toFIGS. 6-8 may be operated to obtain a light field image at an array ofimage sensing regions 900 positioned at a first plane. A syntheticfocusing operation may then be performed on the light field image (e.g.,by constructively and/or destructively combining the pixel values of aset of images included in the light field image) to construct images ofone or more layers 902, 904, 906 of the forearm. An image of one layer(e.g., the layer 906) may include, or bring into focus, the feature 618of the forearm. Images of other layers (e.g., layers 902 and 904) maynot include, or obfuscate, the feature 618 of the forearm.

FIG. 10 shows an exploded view of another set of components 1000 thatmay be included in a biosensor system such as the biosensor system 116shown in FIGS. 2 and 3. The components 1000 may include the cover 606,set of light emitters 602, array of non-overlapping image sensingregions 608, array of light-transmissive elements, and spacer 614described with reference to FIGS. 6-8. However, in the biosensor systemof FIG. 10, the light-transmissive elements may include a set of lenses1002, instead of or in addition to pinholes. In some embodiments, theset of lenses 1002 may be included in a microlens array (MLA).

FIG. 11 shows an exploded view of another set of components 1100 thatmay be included in a biosensor system such as the biosensor system 116shown in FIGS. 2 and 3. The components 1100 may include the cover 606,set of light emitters 602, array of non-overlapping image sensingregions 608, array of light-transmissive elements, and spacer 614described with reference to FIGS. 6-8. However, in the biosensor systemof FIG. 11, the set of light emitters 602 may be intermingled with theset of micro-cameras, and may be positioned to emit light from within aboundary defined by the set of micro-cameras.

In some embodiments of the biosensor systems described with reference toFIGS. 2, 3, 6-8, 10, and 11, the light emitters 602 of a biosensorsystem may include light emitters of different type, such as lightemitters that emit different wavelengths of light. Operation of abiosensor system while simultaneously or sequentially activating lightemitters of different type (e.g., of different wavelengths) can helpdifferentiate features having different light absorption and reflectionspectra. For example, melanin pigmentation can have a very differentabsorption spectrum than blood. Hence, a vascular pattern (e.g., a bloodperfusion pattern in skin) may in some cases be distinguished from apigmentation pattern (e.g., a skin (or melanin) pigmentation pattern)based at least in part on whether the features of the patterns absorb orreflect light of a particular wavelength.

In some embodiments, light emitters may be tuned to emit differentwavelengths of light, or different color filters may be positioned over(or deposited on) different light emitters.

Referring now to FIG. 12, there is shown a watch attached to a forearmof a user near the user's wrist. By way of example, the watch may be thewatch 100 described with reference to FIGS. 1-3. As previouslydescribed, attachment of the watch 100 to the forearm 300 near the wristmay position a first or exterior surface of a cover 114 (i.e., a surfaceexterior to a watch body 102) adjacent a dorsal side of the forearm 300near the user's wrist. However, in contrast to the watch position shownin FIG. 3, the watch body 102 is tilted with respect to the dorsal sideof the forearm 300 in FIG. 12. As a result, the biosensor system 116 andincluded light field camera are tilted with respect to the dorsal sideof the forearm 300.

In some embodiments, the biosensor system 116 may be configured tocompensate for tilt of a light field camera with respect to the dorsalside of the forearm 300. FIG. 13 shows an alternative embodiment of thewatch 100 described with reference to FIGS. 1-3, in which a tilt sensor1300 is included in the watch 100. The tilt sensor 1300 may beconfigured to detect the tilt described with reference to FIG. 12, sothat a processor associated with the biosensor system 116 may compensatefor the tilt. By way of example, the tilt sensor 1300 is shown toinclude a set of proximity sensors (e.g., four proximity sensors 1302).The proximity sensors 1302 may be capacitive sensors, optical sensors,other types of sensors, or a combination thereof.

The proximity sensors 1302 may determine respective distances betweenthe watch body 102 (or a sensing plane of the biosensor system 116) andthe skin of a user's forearm, and may determine a complex angle (e.g., aset of angles in x, y, and z planes) between the sensing plane and theskin.

FIG. 14 shows the conical fields of view 616 of some of themicro-cameras shown in FIGS. 6-8, as well as the feature 618 of theforearm shown in FIG. 8. As described with reference to FIG. 9, thelight emitters and light field camera described with reference to FIGS.6-8 may be operated to obtain a light field image at an array of imagesensing regions 900 positioned at a first plane. However, in contrast tothe synthetic focusing operation described with reference to FIG. 9, thesynthetic focusing operation illustrated in FIG. 14 may be performed onthe light field image (e.g., by constructively and/or destructivelycombining the pixel values of a set of images included in the lightfield image) while compensating for tilt of a sensing plane (i.e., theplane including image sensing regions 900) with respect to a user's skinAlternatively, the tilt may be compensated for prior to performing thesynthetic focusing operation (e.g., by distorting the elemental imagesof the light field image based on the determined tilt). Thus, images maybe constructed for layers 1102, 1104, 1106 of the forearm that intersectthe conical fields of view 616 of the micro-cameras at an angle (i.e.,the images may be normalized so that the images are parallel to thesurface of the skin instead of parallel to the sensing plane defined byimage sensing regions 900). In some embodiments, images may benormalized using a procedure such as shear-warp factorization.

In alternative embodiments, tilt of a biosensor system, light fieldcamera, or sensing plane with respect to the skin of a user's forearmmay be determined using the light field camera. For example, a syntheticfocusing operation may be performed to determine when the surface of theskin (e.g., features on the surface of the skin) comes into focus atsome or all of the micro-cameras in the light field camera. Depthsassociated with the layers at which the surface of the skin comes intofocus may be determined, and the determined depths may be used similarlyto distances determined by proximity sensors to determine the tilt ofthe biosensor system, light field camera, or sensing plane with respectto the skin of the user's forearm.

In further alternate embodiments, a processor associated with abiosensor system including a light field camera may determine a tilt ofthe biosensor system, light field camera, or sensing plane with respectto the skin of a user's forearm, and then: 1) warn the user that thetilt exists, 2) prompt the user to reposition the biosensor system ordevice that includes the biosensor system, and/or 3) instruct the useron how to reposition the biosensor system or device.

A biosensor system, such as one of the biosensor systems described withreference to FIGS. 2, 3, and 6-14, may be randomly positioned in variouslocations on the dorsal side of a forearm near a user's wrist. Forexample, the position of the biosensor system may vary with eachattachment of a device that includes the biosensor system to the user'sforearm, or with the user's adjustment of a watch band or other memberused to secure the device to the user's forearm, or with movement of theuser, or as a result of the user intentionally moving the device to makethe device feel comfortable based on environmental conditions. Toaccount for random positioning of the biosensor system on the user'sforearm, the biosensor system may capture multiple light field imagesfrom different positions during a bioauthentication enrollment operationor health monitoring baselining operation. The multiple light fieldimages may captured at a predetermined time, such as when the device orthe user initiates an enrollment or baselining operation, or in thebackground when the device is worn by the user and not being used forother purposes. The multiple light field images may be used to constructa composite template of the user's forearm. The composite template maybe associated with a larger three-dimensional space than thethree-dimensional space associated with a single light field image.

FIG. 15 shows a sample electrical block diagram of an electronic device1500, which electronic device 1500 may in some cases take the form ofany of the watches or other wearable electronic devices described withreference to FIGS. 1-14, or other portable or wearable electronicdevices. The electronic device 1500 may include a display 1502 (e.g., alight-emitting display), a processor 1504, a power source 1506, a memory1508 or storage device, a sensor 1510, and an input/output (I/O)mechanism 1512 (e.g., an input/output device, or input/output port). Theprocessor 1504 may control some or all of the operations of theelectronic device 1500. The processor 1504 may communicate, eitherdirectly or indirectly, with some or all of the components of theelectronic device 1500. For example, a system bus or other communicationmechanism 1514 may provide communication between the processor 1504, thepower source 1506, the memory 1508, the sensor 1510, and theinput/output mechanism 1512.

The processor 1504 may be implemented as any electronic device capableof processing, receiving, or transmitting data or instructions. Forexample, the processor 1504 may be a microprocessor, a centralprocessing unit (CPU), an application-specific integrated circuit(ASIC), a digital signal processor (DSP), or combinations of suchdevices. As described herein, the term “processor” is meant to encompassa single processor or processing unit, multiple processors, multipleprocessing units, or other suitably configured computing element orelements.

It should be noted that the components of the electronic device 1500 maybe controlled by multiple processors. For example, select components ofthe electronic device 1500 (e.g., the sensor 1510) may be controlled bya first processor and other components of the electronic device 1500(e.g., the display 1502) may be controlled by a second processor, wherethe first and second processors may or may not be in communication witheach other. In some cases, the processor 1504 may perform one or more ofa bioauthentication operation, a bioauthentication enrollment operation,a secure transaction operation, a health monitoring operation, a healthassessment operation, and so on.

The power source 1506 may be implemented using any device capable ofproviding energy to the electronic device 1500. For example, the powersource 1506 may be one or more batteries or rechargeable batteries.Additionally or alternatively, the power source 1506 may be a powerconnector or power cord that connects the electronic device 1500 toanother power source, such as a wall outlet.

The memory 1508 may store electronic data that can be used by theelectronic device 1500. For example, the memory 1508 may storeelectrical data or content such as, for example, audio and video files,documents and applications, device settings and user preferences, timingsignals, control signals, data structures or databases, or referencesets of features used in a bioauthentication, health monitoring, orhealth assessment operation. The memory 1508 can be configured as anytype of memory. By way of example only, the memory 1508 may beimplemented as random access memory, read-only memory, Flash memory,removable memory, other types of storage elements, or combinations ofsuch devices.

The electronic device 1500 may also include one or more sensors 1510positioned almost anywhere on the electronic device 1500. The sensor(s)1510 can be configured to sense one or more type of parameters, such asbut not limited to, pressure, light (e.g., a light field), touch, heat,movement, relative motion, biometric data (e.g., biological images orparameters), and so on. For example, the sensor(s) 1510 may include aheat sensor, a position sensor, a light or optical sensor, anaccelerometer, a pressure transducer, a gyroscope, a magnetometer, ahealth monitoring sensor, a light field camera, and so on. Additionally,the one or more sensors 1510 can utilize any suitable sensingtechnology, including, but not limited to, capacitive, ultrasonic,resistive, optical, light field, ultrasound, piezoelectric, and thermalsensing technology. In some examples, the sensor(s) 1510 may include oneof the biosensor systems described herein.

The I/O mechanism 1512 may transmit and/or receive data from a user oranother electronic device. An I/O device may include a display, a touchsensing input surface, one or more buttons (e.g., a graphical userinterface “home” button), a crown, one or more cameras, one or moremicrophones or speakers, one or more ports such as a microphone port,and/or a keyboard. Additionally or alternatively, an I/O device or portmay transmit electronic signals via a communications network, such as awireless and/or wired network connection. Examples of wireless and wirednetwork connections include, but are not limited to, cellular, Wi-Fi,Bluetooth, IR, and Ethernet connections.

FIG. 16 shows an example method 1600 of authenticating a user of a watchor other wearable electronic device, such as one of the watches orwearable electronic devices described herein.

At block 1602, the method may include emitting light into a dorsal sideof a forearm near a wrist of the user. The operation(s) at 1602 may beperformed, for example, using the biosensor system described withreference to FIGS. 2, 3, and 6-15, the light emitter(s) described withreference to FIGS. 2, 3, 6-8, 10, and 11, or the processor describedwith reference to FIGS. 2, 3, 6-11, and 13-15.

At block 1604, the method may include receiving remissions of the lightfrom the dorsal side of the forearm near the wrist of the user. Theremissions of the light may be received using a light field camera. Theoperation(s) at 1604 may be performed, for example, using the biosensorsystem described with reference to FIGS. 2, 3, and 6-15, the light fieldcamera described with reference to FIGS. 2, 3, and 6-14, or theprocessor described with reference to FIGS. 2, 3, 6-11, and 13-15.

At block 1606, the method may include generating a light field imagefrom the remissions of the light. The operation(s) at 1606 may beperformed, for example, using the biosensor system described withreference to FIGS. 2, 3, and 6-15, or the processor described withreference to FIGS. 2, 3, 6-11, and 13-15.

At block 1608, the method may include performing a synthetic focusingoperation on the light field image to construct at least one image of atleast one layer of the forearm near the wrist. The operation(s) at 1608may be performed, for example, using the biosensor system described withreference to FIGS. 2, 3, and 6-15, or the processor described withreference to FIGS. 2, 3, 6-11, and 13-15.

At block 1610, the method may include extracting a set of features fromthe at least one image. The operation(s) at 1610 may be performed, forexample, using the biosensor system described with reference to FIGS. 2,3, and 6-15, or the processor described with reference to FIGS. 2, 3,6-11, and 13-15.

In some embodiments, the features extracted at block 1610 may include atleast one or two of a hair follicle opening pattern, a hair folliclepattern, a vascular pattern, a vein pattern, an artery pattern, a bloodperfusion pattern in skin, a blood perfusion pattern in tendons, a bloodperfusion pattern in fascia, a tendon pattern, a connective tissuepattern, a skin pigmentation pattern, a small scale folding pattern ofskin, a pore opening pattern, a pore pattern, and a bone shape pattern.In some embodiments, the features may include a relationship between atleast two of a hair follicle opening pattern, a hair follicle pattern, avascular pattern, a vein pattern, an artery pattern, a blood perfusionpattern in skin, a blood perfusion pattern in tendons, a blood perfusionpattern in fascia, a tendon pattern, a connective tissue pattern, a skinpigmentation pattern, a small scale folding pattern of skin, a poreopening pattern, a pore pattern, and a bone shape pattern.

At block 1612, the method may include determining whether the set offeatures matches a reference set of features. The operation(s) at 1612may be performed, for example, using the biosensor system described withreference to FIGS. 2, 3, and 6-15, or the processor described withreference to FIGS. 2, 3, 6-11, and 13-15.

At block 1614, the method may include authenticating the user inresponse to the set of features matching the reference set of features.The operation(s) at 1614 may be performed, for example, using thebiosensor system described with reference to FIGS. 2, 3, and 6-15, orthe processor described with reference to FIGS. 2, 3, 6-11, and 13-15.

In alternative embodiments of the method 1600, the operation(s) at block1614 may include the performance of a health monitoring, healthassessment, payment, banking, or other secure transaction operation. Insome embodiments, the operation(s) at block 1612 may not be performed,and the operation(s) at block 1614 may include a biauthenticationenrollment operation or health monitoring baselining operation.

In some embodiments of the method 1600, the method may further includedetermining a tilt of the light field camera with respect to the dorsalside of the forearm near the wrist, and compensating for the determinedtilt prior to or while performing the synthetic focusing operation, asdescribed for example with reference to FIGS. 12-14.

The foregoing description, for purposes of explanation, uses specificnomenclature to provide a thorough understanding of the describedembodiments. However, it will be apparent to one skilled in the art thatthe specific details are not required in order to practice the describedembodiments. Thus, the foregoing descriptions of the specificembodiments described herein are presented for purposes of illustrationand description. They are not targeted to be exhaustive or to limit theembodiments to the precise forms disclosed. It will be apparent to oneof ordinary skill in the art that many modifications and variations arepossible in view of the above teachings.

What is claimed is:
 1. A watch body wearable electronic device,comprising: a housing; a front side cover mounted to the housing; a backside cover mounted to the housing; a display disposed within the housingand viewable through the front side cover; a light field camerapositioned behind the display and disposed to image a body part of auser through the back side cover; and a processor configured to operatethe light field camera, obtain a light field image of the body part fromthe light field camera, and perform a synthetic focusing operation onthe light field image to construct at least one image of at least onelayer of the body part.
 2. The wearable electronic device of claim 1,wherein the light field camera comprises: an array of non-overlappingimage sensing regions; a pinhole mask positioned between the array ofnon-overlapping image sensing regions and the back side cover; and aspacer between the array of non-overlapping image sensing regions andthe pinhole mask; wherein, the array of non-overlapping image sensingregions is aligned with an array of pinholes in the pinhole mask to forma set of micro-cameras having fields of view that overlap exterior tothe wearable electronic device; and the processor is further configuredto extract a set of features from the at least one image, determinewhether the set of features matches a reference set of features, andperform an operation in response to whether the set of features matchesthe reference set of features.
 3. The wearable electronic device ofclaim 1, wherein the processor is further configured to extract a set offeatures from the at least one image, determine whether the set offeatures matches a reference set of features, and perform an operationin response to whether the set of features matches the reference set offeatures.
 4. The wearable electronic device of claim 3, wherein theoperation comprises a bioauthentication operation.
 5. The wearableelectronic device of claim 1, further comprising: a set of lightemitters; wherein, the light field camera comprises a set ofmicro-cameras having fields of view that overlap exterior to thewearable electronic device; and the set of light emitters is positionedto emit light from within a boundary defined by the set ofmicro-cameras.
 6. The wearable electronic device of claim 1, furthercomprising: a light emitter positioned to emit light through the backside cover.
 7. A wearable electronic device, comprising: a housinghaving a front side and a back side; a light emitter positioned withinthe housing and configured to emit light through the back side of thehousing into a body part of a user; a light field camera positionedwithin the housing and configured to receive remissions of the lightthrough the back side of the housing; and a processor configured toobtain at least one image of the body part from the light field camera.8. The wearable electronic device of claim 7, further comprising: a setof proximity sensors; wherein, an output provided by the set ofproximity sensors triggers the processor to obtain the at least oneimage of the body part from the light field camera.
 9. A method ofauthenticating a user of a wearable electronic device, comprising:emitting light into a body part of the user; receiving, using a lightfield camera, remissions of the light from the body part of the user;generating a light field image from the remissions of the light;performing a synthetic focusing operation on the light field image toconstruct at least one image of at least one layer of the body part;extracting a set of features from the at least one image; determiningwhether the set of features matches a reference set of features; andauthenticating the user in response to the set of features matching thereference set of features.
 10. The wearable electronic device of claim1, wherein the processor is further configured to extract a bloodperfusion pattern from the at least one image.
 11. The wearableelectronic device of claim 10, wherein the blood perfusion patterncomprises at least one of: a blood perfusion pattern in skin; a bloodperfusion pattern in tendons; or a blood perfusion pattern in fascia.12. The wearable electronic device of claim 1, wherein the processor isfurther configured to extract a vascular pattern from the at least oneimage.
 13. The wearable electronic device of claim 1, wherein theprocessor is further configured to extract a pigmentation pattern of thebody part from the at least one image.
 14. The wearable electronicdevice of claim 1, wherein the processor is further configured toextract a pore pattern from the at least one image.
 15. The wearableelectronic device of claim 1, wherein the processor is furtherconfigured to perform a bioauthentication of the user of the wearableelectronic device, the bioauthentication performed using the at leastone image.
 16. The wearable electronic device of claim 1, wherein: theprocessor is further configured to, extract a set of features of thebody part from the at least one image; and classify different featuresin the set of features based on at least one of feature size, featuredepth, feature color, feature movement, feature variance between layers,or feature presence in multiple layers.
 17. The wearable electronicdevice of claim 6, wherein the light emitter comprises an infraredemitter.
 18. The wearable electronic device of claim 6, wherein thelight emitter is configured to emit light into the body part.
 19. Thewearable electronic device of claim 7, wherein the processor is furtherconfigured to perform a health monitoring operation using the at leastone image of the body part.
 20. The wearable electronic device of claim7, wherein: the processor is further configured to, extract a set offeatures from the at least one image; determine the set of featuresmatches a reference set of features; and perform a health monitoringoperation in response to the set of features matching the reference setof features.
 21. The wearable electronic device of claim 7, wherein theprocessor is further configured to provide, to the user, feedback on afitness level of the user, the feedback determined using the at leastone image of the body part.